A real-time silent speech system for voice restoration after total laryngectomy / Un sistema de voz silenciosa en tiempo real para la restauración de la voz después de la laringectomía total

Background and aim: Individuals who have lost their voice following a laryngectomy as a treatment for cancer will inevitably struggle with their daily communication. Unfortunately, the current methods for speaking after laryngectomy all have limitations, either because of the poor acoustics generated by these methods or because they are potentially harmful. The aim of this work is thus to explore an alternative method for post-laryngectomy voice restoration in which the movement of the intact articulators is captured and then converted into audible speech using machine learning techniques. Materials and methods: To demonstrate the feasibility of speech generation from captured articulator movement, 6 healthy adults were recruited. For each subject, both the speech acoustics and the subject's articulator movements were recorded simultaneously. Articulator movements were captured using a technique known as permanent magnet articulography (PMA), in which small magnets are attached to the articulators (typically tongue and lips) and the magnetic field generated by the magnets is captured with sensors located close to the mouth. Deep artificial neural networks were then used to model the mapping between the sensor data and the speech acoustics, thus, enabling the synthesis of speech from captured articulatory data. Results: The proposed silent speech system is able to generate speech that sounds natural, resembles the subject's own voice and is fairly intelligible (up to 92% intelligibility for some speakers on a phonetically-rich corpus). Conclusions: With further research, the proposed system could in future be a real option to restore lost voice after laryngectomy

Antecedentes y objetivo: Aquellas personas que han perdido su voz después de una laringectomía se ven limitadas irremediablemente en su comunicación diaria. A pesar de existir en la actualidad métodos para recuperar el habla tras la laringectomía, todos ellos presentan limitaciones. El objetivo de este trabajo es explorar un método alternativo para hablar tras la laringectomía, en el que el movimiento de los órganos de la voz se transforma en una señal acústica utilizando técnicas de aprendizaje automático. Materiales y métodos: En esta investigación participaron 6 adultos sanos. Para cada sujeto se grabó tanto su voz como los movimientos de sus labios y lengua. Los movimientos de los órganos del habla fueron capturados usando una técnica conocida como Articulografía de Imán Permanente (PMA), en la cual pequeños imanes se colocan sobre estos órganos y el campo magnético generado por los imanes se captura usando unos sensores sensibles al campo magnético. Se utilizaron redes neuronales artificiales profundas para modelar la transformación entre los datos de los sensores y la acústica de la voz. Resultados: El sistema de habla silenciosa propuesto es capaz de generar voz que suena natural, se asemeja a la propia voz del sujeto y es inteligible (hasta un 92% de inteligibilidad para algunos sujetos). Conclusiones: El sistema propuesto podría ser en el futuro una opción viable para restaurar la voz tras una laringectomía total

Restoring Speech Following Total Removal of the Larynx.

By speech articulator movement and training a transformation to audio we can restore the power of speech to someone who has lost their larynx. We sense changes in magnetic field caused by movements of small magnets attached to the lips and tongue. The sensor transformation uses recurrent neural networks.

A voice-input voice-output communication aid for people with severe speech impairment.

A new form of augmentative and alternative communication (AAC) device for people with severe speech impairment-the voice-input voice-output communication aid (VIVOCA)-is described. The VIVOCA recognizes the disordered speech of the user and builds messages, which are converted into synthetic speech. System development was carried out employing user-centered design and development methods, which identified and refined key requirements for the device. A novel methodology for building small vocabulary, speaker-dependent automatic speech recognizers with reduced amounts of training data, was applied. Experiments showed that this method is successful in generating good recognition performance (mean accuracy 96%) on highly disordered speech, even when recognition perplexity is increased. The selected message-building technique traded off various factors including speed of message construction and range of available message outputs. The VIVOCA was evaluated in a field trial by individuals with moderate to severe dysarthria and confirmed that they can make use of the device to produce intelligible speech output from disordered speech input. The trial highlighted some issues which limit the performance and usability of the device when applied in real usage situations, with mean recognition accuracy of 67% in these circumstances. These limitations will be addressed in future work.